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ABSTRACT 

A repo.rt is given on two years' experience in using 
the^TlTtrfKiX Test Study (ATS) norms, " de veloped by the Educational' 
Testing Se^iee. under cont'rafct to the U.S. Officer of Educatian as 
part of the Washingtbn .State Assessment program. In the fitst year, 
the desire to deve.lop a state reading achievement profile through the 
application of the ATS norm tables was incorporated into the 
Washington State ESEA T^tle Ill'^nfeeds assessment pia,n fop fiscal year 
1974. A 20% sample of sc]i6ols containing grade 6 was drawn, and 
attempts made to collect sixth-grade test datta compatible with the 
ATS tables. The data wete tabulated and reported as total 'reading 
mean .•scores i^and standard deviations for the state as a whole and for 
t4n categories based on district "size. Because of sampling and data 
collection problems, changes were made in xhe second year's endeavor. 
All districts using tests covered by the ATS norms werjs given the 
opportunity to contribute data. No attempt was. made to generalize 
beyond the population supplying test results, ' but the , analysis waS; 
Expanded to, j^nclude the reading subtests and grades 4 and 5 as well 
as grade 6. '(Author) * , ' ' %' ^ 
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Abstract ' ; * 

This paper reports on two years* experience in u^ing the Ancho^r Test 
Study (^TS) norms, developed by Educational TesCring Service under contract td 
ther U. S. Office of Education, as part of the Washington State Assessment pro- 
gram. In the' first y^ar, the desire to develop a. state read ing^, achievement 
prof ile "through^ the application cjf th^ ATS norm tables was incorporated lifto, 
the Wd^shington State ESEA Title III needs assessment plan for fiscal y6ar 1974. 
The support for this style of assessment rested on both' the^ interest in. 
generating a description of the reading performance of WashingttDn ;pupils and' 
on the interest in studying the feasibility of using the ATS nprms and local 
school district test data as the basis for constructing a state profile of 
reading achievement. A twenty percent sample of schools containing grade six 
was drawn and attempts made to collect* sixth grade test data compatible with^' 
the ATS tables. The resulting data were tabulated and reported in the form of 
total reading mean scores, and standard deviations for the state as a whole dnd 
for ten categories based on district size. , . 

Because of the sampling' and data collection ptoble^ms eticoiintered in the 
first application of the Anchor Test Study, changes were made in the second year's 
endeavor. Rather than attempting to achieve a representative state prof ile ,of 
reading achievement l^ogr one grade level, based on a random sample, all di'stricts 
using tests covered by the ATS norms were given the opportunity to contribute data. 
No attempt, was made to geheraliae beyond the population supplying test results, 
but the analysis w^s expanded to include tUe reading sub~tests and grades four 
and five as well as gradp six. . ^ . 
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ASSESSING READING ACHIEVEMENT USING THE 
ANCHOR TEST STUDY 



PART I 



. ! 



Thor desire to develop state reading achievement profile through the 
application of the ATS ■ tables was incorpbrated into the Washington State 
ESEA Title III Needs Assessment Plan for fiscal year 197^. The support for* 
this style of assessment rested oii both the interest in feeneratitig a descrip- 
tion of the reading performance of Washington pupils and on the interest in 
the feasibility of using the ATS norm tables and local school iisttict*-<Lata 
as the- basis for constructing a state profile of reading achievement. A 
listing of the tests included in the complete tables is given below. 



Tut edition 


} 


1 # ^ ^ r 

^tvtl us«d at gradt 


Form 


4 




6 


0 

Cahf Achievement Tests 
(197bed) ^ 


A 


3 


: ' ,3 


^ — 

4 


Compfehensive Tests of 
Basic Skills (1968 cd ) 


Q 


2 


2- 


3 " 


Gate* M.KGmitm Reading 
Tests (1964 cd) 


1M 


Survey D 


Survey D 


Survey 0 


iowa Jests of Basic Skills 
(197t ed) 


5 


10 


11 


. 12 


Metropolitan Achievement 
Tests (1070 ed ) 


F 


Eiemen 
tary 


Inter- 
mediate 


Inter* 
mediate ^ 


Sequenhjl Tests.o^ , 
Educationaf Progress. 
STEP Series M 11969 ed ) 


A 


4 


4 

t 


4^ * 


SRA Achievement 
Series (1971 ed) 




. Blu»» 
ediiix>n 


Blue 
edition 


Gree*i 
edition 


Stan (ord Achievement 
Tests (1964* ed 1 


w 


mediate I 


Inter- 
mediate II 

4-, 


Jnte' 
me<fia!e n 



- \<[\Qxx the Anchor Test Study Users Manual (unofficial version not 
.including the Gates-MacGinit ie) became availabt^to the Washington Superin-*^K 
tendent of Public Instruction in the fall of -197'3/, the imp lament at iWtff: 
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the plan began. State "files*^ were inspected to determine the availability 
of information regarding local school district use of standardized tests. 
Since no systematic records existed, a questionnaire (Anchor Test Profile 
Survev^ was prepared and sent to each school district to collect information 
related to their, use of, the tests included in the ATS tables. 

/This instrument was retuimed by approximately 80% of the local s.chool' 
districts and served as the basis for two decisions. First, grade six was 
selected for analysis because more of the ATS tests were administered at 
that grade than at the fourth grade or the fifth. Second, the data were 
used to check the feasibility of the sample units drawn and to^ndicate where 
alternatives would be required. ^ 

In an effort to generalize to the state as a-whole. and to ten size 
categories a^stratified random sample was designed. Each common school con- 
taining; grade six was assigned to one of ten categories based on total digtrict 
enrollment. Using the school as the sample unit, 20% were drawn randqmly from 
each size category. An additional 10% sample of schools was drawn to provide 
alternates. As the selected schools' fro^i the 20% drawing were compared with 
' the results of the questionnaire,, the alternates were used to fill in for ' 
those schools located in districts in which the testing programs were^not 
compatible with the Anchor Test Tables ©r in districts which did not return 
the quesi;i<^«xn£iii'e. In an effort to maintain the number'of schools required 
in each size'^&^tegory for a 20% sample, the majority of" the alternates were 
used. In two instark^s, all of the alternates- v^ere substituted for original 
selections * 
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Requests were sent to the school districts for the raw scores of sixth 
grade students attending the sample schools^. The data submitted by the school 
districts were studied and forwarded to the Nortj^west Regional Educational" 
Laboratory for tabulation. Three circumstances became apparent. First, 
several districts did not complete the Anchor Test Profile Survey accurately 
and did not possess the information as claimed. Second, the test results 
were submitted in a greater variety of forms >than anticipated, especially 
with reference to the style of reporting; for example, percentiles, stanines i 
.^rade equivalencies, growth scores, as well as raw s c o re s were reported and 
the time^ of test adminijltr^i_ti5jx--€^veTi^9^^ from SepTn!llTfeir*»fe©.**June. 

Third, it wculd be impossible to maintain a 20% randoiir^.ainple in each of the 
ten size categories a? -^^ basis for generalization. (See th^ t^ble below 
for the actual sample and number of schools and students included in the 
study.) 




District sue ftod Sample Sire Used In the 
Anchor Test StucJy Data Collection Effort 
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Stratum 


District 


Nmrfier of Schools 


Percent 


Number of 


COQo 


Size 


in Sample 


In Sample* 


Students 


a 


20,000 and over 


23 


10.2 


17^7 


b 


10.000 ' 19.999 


17 


10.5 


1469 


c 


5,000 - 9,999 


15 


10.6 


951 


d 


3.000 - A, 999 


6 


7.7 


856* 


\ 

e 


2.000 - 2,999 


5 


10. A 


764 


f 


1.000 - 1.9'19 ' 


5 


8.5 


252 




700 -*999 


\ 


11.8 


229 


h 


500 - 699 


3 


10.7 


149 


1 


300 - A9^ 


3 




85 


j 


under 300 


6 


6.5 


66' 


TOTALS 


87 * 




6568 



Note.— District ond School Slit estlnatei were based on 1972 School Ccnauj 
data for the State of Waahlngton. 
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, 'Hie problems of data analysis were greatly increased in an effort to 
maintain some semblance of a random sample, and in many inst^n'ces precision in 
dealing with the lack of compatibility in test forms > levels, editions and in 
time of test administration suffered as a consequence. The issue of sampling 
^nd the power to generalize is crucial in this day of full disclosure when 
both the public and the pres^ ^demand access to information regardless of its 
quality or of the assumptions violated by generalizing. * 

Although t^^e 1973-74 study , resulted in a somewhat limited description of 
reading performance, the activities did lead to a successful feasibility s.tudy 
highlighting a number of, trouble spots and pitfalls to guard against. The 
results of the reading assessment are displayed in the following table. 



\ 



EflCltnated SCaCt «fid 5tr«t« Means and Standard Deviations for 
Six Standardized Tests. Grada Six Total Reading Scores. Spring 1972 



Stratum 



Standardized Reading Tests 



Code 


CTBS 


ITBS ..^ 


MAT 


SAT 


SRA 


STEP I„I 
















a 




61,6 


63.5 


62,3 


53.6 


42.1 




b.6 


8.9 


7.6 


8.8 


7.9 - 


4.5 


b 


43.1 


57.0 


60.0 


57.8 


49.7 


40.1 




7.8 


10.6 


9.2 • 


10.6 


9.6 


5.6 


c 


49.6 


65.8 


67.3 


66\6 


57.4 


44,4 




4.9 


6,8 


5.2 


6.5 


5.7 


3.0 


d 


48.9 


64,7 


66.0 


65,0 ' 


55.6 


43.4 




4.9 


7,2 




. 6,6 


5,5 


2.7 




52.9 


70.7 


70,5 


7a. 9 


61.0 


46.2 




7.1 


10.2 


6.9 


9>^ 


8.1 


4.0 




AG. 6 


61.9 


64,2 


62. 6\ 


54.5 






4.6 


6.4 


5.3 


6.2 \ 


5.6 


3.1 


1 


4l6 


58.3 


60.9 


59.4 ' 


\50.5 


40,6 




6.2 


11.7 


^ 9.6 


12.0 


N^p.3 


5.7 


h 


46. A 


61.4 


64.0 


62.2 


53M 


43.1 




3.3 


4.5 


3.6 


4,3 


4.4 


0.9 


1 


41,9 


55.8 


57.4 


50.2 


46.2 


38.3 




17.7 


22.9 


23.1 


18.5 


23.7 


14.2 


J 


50.2 


67,1 


67.5 


66.5 


58,4 


44.9 




8,4 


11,4 


9.8 


10.7 


9.9 


6.8 


st«t# 


47. 0 


62.4 


64.1 


62.4 


55.0 


42.6 


(All Schools) 


7.4 




8.6 


9.4 


9.1 


5.1 



Note.— First nuwbor reptasects the n«nn. Second number represents the 

standard deviation. 

CAT state and strotuo mettui con be estlmoted from the data 
using Educ«)tlonal Testing Service equivalency tables. The CAT means 
foi strata a-j nnd thit state resptctlvely ore approximately as follows: 
44, 40,5 , 46,5. 45. 5, 50. 44. 41, 44, 39. 47. and 44. 
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PART II ^ 

The findings of the first study were sufficient to encourage the further 
use of the ATS tables on the state level, and it was decided to design a 
second study pursuing a different approach. In order to gain a more complete 
description of district testing activities, an addendum page was added to 
the basic program report submitted by all districts in order to qualify for 
general state education funds. In this yay a profile of each district's 
standardized testing -program was obtained as a more accurate starting point 
for decision making. This information revealed that if both fall and spring 
testan^ were included, that at any one grade level leSs than one-third of 
the districfta administered an ATS test' in a form compatible with the tables., 
and less than one-half of the districts administered a usable test if all 
three of the grade levels were considered together. 

This analysis, add^d to the finding^ of the feasibility study, curtailed 
all attempts to design and draw a random sample. Instead, all districts 
reporting a compatible tei&ting program were asked to voluntarily^ submit 
student raw scores for inclusion in Xhe calculation 6f "norms" in the form 
of means and standard deviations for each of the tests. The merit of this 
activity is in providing a service to local school districts. Since there is 
no way of ^;uarantecing the representativeness of the districts and schools 
submitting scores, there is no way of generalizing with confidence to a state 
profile at e4ch grade level. The norms created will be norms for educational 
units making use of one of the eight ATS tests rather than true state norms 
in the traditional sense. ThJLs endeavor takes on an added dimension, however 
wh^n it is noted that although less than 50% of the districts use appropriate 
Anchor Tests, the/ provide educat:^on for the vast majority of children in 

■ ■ 7 ■ : ' ■ • ■ 
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the State of Washington. The goal is to make it possible for districts jjsing > 
any of the Anchor Test? to compare their results with the norms established 
by all districts administering the tests and to aggregate achievement data 
generated from the use of more than one of the tests included in the Anchor 
Study. 

To implement this second phase endeaVor, jnaj^r emphasis was placed on 
the development of computer programs to facilitate the use of the ATS tables 
£or both state and local assessment purposes. The Northwest Lab was con- 
tracted to assist in developing the computer programs to provide score 
transformation among the eight tests and conversion between ^f all , winter , 
and spring norms. Tlie resulting programs accomplished three key purposes. 
First, the ATS equivalency tables were computerized so that test scores can 
be equated with speed. However, since the original ATS tables report only 
raw scores and spring norms, they are of limited use for large scale assess- 
ments b"^sed on existing data. To provide greater flexibility, two additional 
steps were taken. Second, tables were developed and programmed to convert 
fall and winter testing times to spring norms. Third, tables were also pro- 
).;rammed to convert the standard reporting options, for example, grade equivalent 
scores, percent?iles , and scale score.s, to raw scores. ^ • ^ , » 

Tliese programs increase the practical utility of the original ATS 

accomplishments. The Northwest L^b used the following example to summaxize 

the usefulness. 

For example. School A may report grade equivalent scores 
from Fall testing with the California Achievement Tests r 
wljile School B reports raw scores for the Same time axid 
test. School C may use Spring percentile's from the Iowa 
Tests of Basic Skills, while School D has Spring raw 
scores for the Stanford Achievement Tests. By using the- 
Anchor Tejst Program, these schools can now communicate 
meaningfully with each other about these test scores. 

. : 8 
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Efforts are now underway to make the Anchor Test Program available 
to those Washington school districts and other common school district agencies 
having computer installations. 

The availability of the ATS tables, norms for schools using "Anchor 
Tests," computer programs for ease of conversion, and state technical reports 
and experien*^e should encourage district personnel to give greater attention 
to the use of vreading achievement data in local assessments and prograp' 
evaluations. The state experience should also encobrage '^ocal personnel to 
give special consideration to the flexibility "provided by. the original ATS 
norm tables and the Anchor Test computer programs without lod^n^ the ability 
to aggregate scores across projects. 
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